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EXECUTIVE  SUMMARY 


If  we  are  to  present  instruction  that  is  available  anytime  and  anywhere,  takes  advan¬ 
tage  of  the  substantial  tutorial  efficiencies  of  one  teacher  for  every  student,  and  is  affordable, 
we  must  have  recourse  to  technology— specifically,  computer  technology.  Such  technology 
can  be  used  in  instructional  applications  that  range  from  drill  and  practice  and  tutorial  dia¬ 
logues,  to  multiplayer  simulations  and  games.  It  can  be  used  in  stand-alone  modes,  or  it  can 
be  used  to  supplement  classroom  instruction.  It  can  be  used  by  individuals  or  by  groups.  In 
all  cases,  however,  it  must  take  account  of  the  current  state  of  the  learner,  the  eventual  state 
of  the  learner  that  the  instruction  is  intended  to  produce,  and  the  instructional  techniques  that 
reliably  effect  transitions  from  one  state  to  the  other. 

Models  of  the  learner  that  represent  these  current  and  objective  states  must,  to  an 
appreciable  extent,  be  models  of  the  learner’s  cognition,  something  that  underlies  compe¬ 
tence  and  produces  the  human  skills,  performance,  and  abilities  needed  for  success  in  all 
military  operations.  These  models  can  be  implicit,  as  in  intrinsically  programmed  instruc¬ 
tion,  or  they  can  be  explicit.  Both  types  have  been  used  in  technology-based  instruction 
from  its  beginning. 

Early  explicit  models  were  largely  quantitative.  These  models  involved  relatively 
simple  instructional  paradigms  but  fairly  elaborate  mathematics,  including  instructional 
applications  of  optimal  control  theory.  Current  efforts  are  more  concerned  with  qualitative 
models,  19  of  which  are  briefly  described  and  discussed.  Each  of  these  models  can  con¬ 
tribute  to  the  efficiency  and  effectiveness  of  technology-based  instruction.  However,  new 
challenges  have  arisen  from  today’s  uncertain,  asymmetric  operational  environment,  and 
these  challenges  may  require  responses  that  cannot  be  foreseen  or  be  well  prepared  for  in 
advance. 

Our  military  personnel  must  be  prepared  to  expect  the  unexpected  and  to  meet 
unexpected  events  with  individual  and  collective  agility,  creativity,  and  adaptability.  These 
qualities  are  fundamentally  cognitive  in  nature  and  require  more  powerful  and  comprehen¬ 
sive  cognitive  models  if  they  are  to  serve  our  programs  of  education,  training,  decision¬ 
making,  and  performance  aiding  successfully. 
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MODELING  THE  USER  FOR 
EDUCATION,  TRAINING,  AND  PERFORMANCE  AIDING 


A.  INTRODUCTION 

This  document  addresses  research  on  digital  representations  of  human  cognitive 
processes  that  can  be  used  to  develop  computer-mediated  learning  and  performance-aiding 
systems.  We  refer  to  such  representations  as  “models”  of  human  cognition.  Since  this 
topic  is  extensive  in  breadth  and  depth,  we  focused  our  discussion  on  the  following 
questions: 

•  What  is  the  military  value  of  these  models? 

•  What  is  their  current  state  of  development? 

•  What  is  their  relationship  to  instructional  systems  development? 

•  What  research  and  development  (R&D)  should  be  undertaken  to  advance  their 
value  and  utility? 

Many  valuable  contributions  are  being  made  by  researchers  who  are  modeling  neu¬ 
rological  activity  at  one  end  of  the  behavioral  spectrum  and  by  researchers  who  are  mod¬ 
eling  human  physical  activity  and  performance  at  the  other  end.  This  document  is  aimed 
somewhere  in  the  middle  of  these  efforts.  It  concerns  models,  or  representations,  of  human 
cognitive  processes  such  as  perception,  memory,  learning,  decision-making,  and  problem 
solving.  These  processes  arise  from  “micro”  neurological  activity  at  one  end  of  the  spec¬ 
trum  and,  in  turn,  produce  “macro”  physical  activity  and  performance  at  the  other  end. 
Eventually,  R&D  may  yield  models  that  unify  the  full  spectrum  of  human  behavior,  from 
neurons  to  psychomotor  activity;  however,  the  current  state  of  knowledge  limits  us  to  efforts 
to  understand  and  model  components  of  this  spectrum.  Hence,  this  document  focuses  on 
one  such  component,  cognition,  which  seems  an  appropriate  level  of  concern  for  learning 
and  human  performance. 

The  teim  “the  user”  is  used  throughout  this  document.  This  term  is  intended  as  a 
catch-all  for  students,  decision-makers,  technicians,  analysts,  and  anyone  else  who  may  be 
using  computer  technology  for  education,  training,  decision-making,  and  performance 
aiding.  Our  focus  is  on  the  digital  representation  of  these  users’  cognitive  processes. 
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B.  THE  MILITARY  VALUE  OF  COGNITIVE  MODELING 


An  obvious  and  frequently  neglected  fact  is  that  human  competence,  which  is  a 
product  of  human  cognition,  is  essential  to  every  military  operation  across  all  echelons  of 
command  and  activity.  Its  importance  is  perennially  evident  in  the  conduct  of  military 
operations.  Even  in  the  increasingly  technology-saturated  environments  of  modem  opera¬ 
tions,  human  competence  is  needed  to  launch  and  control  systems  in  space,  to  operate  and 
maintain  robotic  vehicles,  to  deploy  remote  sensors  and  systems  in  contested  territory,  and 
so  forth.  In  short,  we  have  no  unmanned  systems.  Without  competent  people  to  operate, 
maintain,  and  deploy  our  materiel  assets,  investments  in  these  assets  will  return  little  and 
may,  in  fact,  be  wasted.  Given  the  wide  availability  of  technology  and  the  ease  with  which  it 
can  be  obtained,  human  competence  may  increasingly  account  for  the  difference  between 
success  and  failure  in  military  operations.  Its  availability  to  commanders  anytime  and 
anywhere  is  a  matter  of  first  importance. 

How  can  we  ensure  this  availability?  Training  (and  its  performance-aiding  analog) 
provides  one  means  to  accomplish  this  objective,  particularly  if  this  training  can  be  delivered 
anytime  and  anywhere.  For  example,  we  might  supply  each  user  with  an  omnipresent  tutor. 
Such  tutoring  is  probably  best  done  by  a  human  who  possesses  expertise  in  the  relevant 
subject  matter,  by  a  comprehensive  range  of  tutorial  techniques,  and  by  sufficient  knowledge 
of  the  user  to  identify,  establish,  and  sustain  in  that  individual  the  precise  human  competence 
needed.  Research  has  shown  such  tutoring  to  be  extremely  effective,  producing  an  often- 
noted  two  standard  deviations  of  improvement  over  less  accessible  and  less  effective  class¬ 
room  instruction  (Bloom,  1984).  However,  such  an  approach  has  also  been  called  an 
instructional  imperative  and  an  economic  impossibility  because,  for  obvious  reasons,  a 
human  tutor  cannot  be  supplied  to  every  user.  This  situation  creates  a  gap  between  what  is 
needed  and  what  we  can  afford.  As  in  many  other  endeavors,  technology  is  being  applied  to 
fill  this  gap. 

The  research  evidence  suggests  that  such  applications  of  technology  can  succeed.  In 
nearly  300  studies  comparing  classroom  (one  teacher,  many  students)  with  computer-medi¬ 
ated,  individualized  instruction  (one  computer,  one  student)  across  many  different  settings 
and  subject  matters,  a  “rule  of  thirds”  emerges.  That  is,  compared  with  classroom  instruc¬ 
tion,  technology-based  instruction  costs  about  a  third  less  and  also  either  increases  achieve¬ 
ment  by  a  third  when  instructional  time  is  held  constant  or  decreases  time  to  reach  constant 
levels  of  achievement  by  about  a  third.  Fletcher  (1997,  2002),  Foster  and  Fletcher  (2002), 
Kulik  (1994),  Niemiec,  Sikorski,  and  Walberg  (1989),  and  others  have  presented  more 
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detailed  discussions  of  these  data.  The  primary  payoff  for  military  operations  is  the  more 
rapid  and  reliable  preparation  of  personnel  to  perform  operational  duties,  which  produces 
significant  payoffs  for  resource  expenditures,  readiness,  and,  most  importantly,  operational 
effectiveness. 

Similar  research  evidence  exists  in  support  of  technology  used  to  aid  performance 
and  decision-making.  For  instance,  technicians  with  only  general  training  have  been  found 
to  perform  as  well  as  specialists  (who  required  time-consuming  and  expensive  training)  if 
they  are  provided  hand-held  or  wearable  performance  aids  (e.g.,  Fletcher  and  Johnston, 
2002;  Joyce,  2001;  Wisher  and  Kinkaid,  1989).  These  aids  contribute  to  military  readiness 
and  effectiveness  not  only  by  enabling  individuals  to  be  released  earlier  for  operational  duty, 
but  also  by  improving  human  competence  for  maintaining,  operating,  and  deploying  materiel 
assets— thereby  significantly  improving  materiel  readiness. 

Costs  are  also  a  factor.  For  instance,  the  United  States  military  spends  about 
$4  billion  a  year  on  specialized  skill  training.  This  is  the  training  provided  after  “basic”  or 
accession  training  to  qualify  personnel  for  the  many  technical  jobs  (e.g.,  wheeled  vehicle 
mechanics,  radar  operators,  avionics  technicians,  oceanographers,  medical  technicians) 
needed  to  perform  military  operations.  It  does  not  include  aircraft  pilot  training,  field 
training,  or  factory  training,  which  are  covered  in  separate  cost  categories.  If  the  United 
States  were  to  reduce  by  30  percent  the  time  to  train  20  percent  of  the  personnel  undergoing 
specialized  skill  training,  it  would  save  over  $250  million  per  year.  If  it  were  to  do  so  for 
60  percent  of  the  personnel  undergoing  specialized  skill  training,  it  would  save  over 
$700  million  per  year  (Foster  and  Fletcher,  2002).  These  are  appreciable  savings  by  most 
standards. 

What  do  these  analyses  and  observations  have  to  do  with  cognitive  modeling? 
Effective  education  and  training  must  start  with  a  dynamic  and  updateable  understanding,  or 
model,  of  the  current  state  of  the  user— a  model  of  the  knowledge,  skills,  and  abilities  the 
user  should  attain  and  the  instructional  techniques,  strategies,  and  processes  needed  to  meet 
the  goals  of  this  model.  This  sort  of  modeling  occurs  in  classroom  learning  where  teachers 
continually  assess  what  their  students  know,  the  level  or  degree  to  which  they  know  it,  and 
the  most  efficient  ways  to  progress  in  achieving  instructional  goals.  As  discussed  in  the  next 
section,  this  modeling  is  also  found  both  implicitly  and  explicitly  in  effective  technology- 
based  instruction. 

Similar  modeling  processes  are  also  required  to  support  performance  aiding  and 
decision-making,  even  though  the  emphasis  in  these  applications  is  on  problem  solution 
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rather  than  on  learning.  What  is  needed  is  a  model  of  the  user  to  provide  advice  that  can  be 
understood  and  carried  out,  a  model  of  the  system  or  situation  with  which  the  user  is  inter¬ 
acting,  and  the  ability  to  maintain  something  similar  to  an  instructional  dialogue  to  help  the 
user  identify  correct  solutions  or  decisions. 

In  both  cases,  concern  with  the  knowledge,  skills,  and  abilities  that  comprise  human 
competence  leads  us  to  human  cognition  and  the  need  to  map  current  cognitive  states  onto 
goal  cognitive  states  and  determine  what  must  be  done  next.  This  presents  severe  difficulties 
for  classroom  instruction  (one  teacher,  many  students).  For  instance,  a  problem  arises  from 
the  degree  to  which  students  in  a  typical  classroom  vary  in  their  prior  knowledge,  abilities, 
and  learning  progress.  Research  suggests  variation  by  about  a  factor  of  five  (e.g.,  Corno  and 
Snow,  1986;  Gettinger  and  White,  1980;  Gustaffson  and  Undheim,  1996;  Tobias,  1989). 
Especially  important  for  military  applications  is  the  observation  that  variability  in  prior 
knowledge  increases  with  age  and  may  be  more  important  in  determining  progress  in  post¬ 
secondary  venues  such  as  military  training  than  it  is  for  students  in  their  earlier  years  of 
schooling.  For  both  efficiency  and  effectiveness,  such  variability  suggests  the  importance  of 
tailoring  interactions  in  military  education,  training,  and  performance  aiding  to  the  specific 
needs  of  individual  users. 

Assessment  of  cognition  in  classroom  instruction  is  necessarily  both  informal  and 
imprecise.  If  we  seek  to  achieve  human  performance  outcomes  reliably  (anytime,  anywhere) 
and  affordably,  we  must  have  recourse  to  technology.  If  we  are  to  use  computer  technology 
to  achieve  these  ends,  we  must  be  able  to  represent— or  model  in  digital  form— current  cog¬ 
nitive  states,  goal  states,  and  ongoing  progress  from  one  to  the  other.  The  empirical  results 
discussed  previously,  arising  from  technology-based  education,  training,  and  performance 
aiding,  suggest  to  some  degree  that  we  have  been  successful  in  doing  this.  The  question 
then  naturally  arises  as  to  how  well  our  ability  to  implement  and  use  such  models  meets  the 
need  for  them. 

C.  THE  CURRENT  STATE  OF  COGNITIVE  MODELING 
1.  Implicit  Cognitive  Models 

Cognitive  models  are  implemented  implicitly  and  explicitly  in  technology-based 
instruction.  Consider  the  following  sample  instructional  item,  which  is  typical  of 
much— perhaps  most— computer-mediated  instruction: 
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In  the  multiplication  problem  3x4=12,  the  number  12  is  called  a 


A. 

Factor 

{Branch  to  remedial  XI} 

B. 

Quotient 

{Branch  to  remedial  X2} 

C. 

Product 

{Reinforce,  go  to  next} 

D. 

Power 

{Branch  to  remedial  X3} 

In  this  item,  the  system  (the  computer  instructor)  assumes  that  a  student  responding 
“A”  misunderstands  the  meaning  of  “Factor”  and  lacks  an  understanding  of  “Product,” 
or  both.  The  student  will  be  branched  to  some  instructional  materials  intended  to  correct  one 
or  the  other  of  these  cognitive  states  and  then  will  be  returned  to  this  item  or  one  similar  to 
it.  The  same  type  of  remedial  approach  is  applied  to  responses  of  “B”  and  “D.”  A  student 
responding  with  “C”  may  be  rewarded  (i.e.,  reinforced)  with  positive  feedback  and  is  then 
sent  to  whatever  item  will  continue  progress  toward  the  instructional  goal(s)  — an  action  that, 
by  itself,  may  constitute  positive  reinforcement. 

The  preceding  example  appears  in  an  article  by  Norman  Crowder  written  for  auto¬ 
mated  teaching  (Crowder,  1959).  We  can  assume  that  the  use  of  cognitive  models  is  a  not 
recent  innovation  in  technology-based  instruction,  but  a  model  of  cognition  and  instructional 
progress  is  evident  in  this  approach.  It  covers  transitions  from  unlearned  to  learned  states 
and  illustrates  what  Crowder  called  “intrinsic”  programming.  This  approach  stands  in 
contrast  to  the  expensive  and  difficult-to-prepare  “extrinsic”  programming  advocated  by 
B.  F.  Skinner  (e.g.,  1954),  and,  for  reasons  of  economy  and  utility,  it  is  the  dominant 
approach  still  in  use  today  in  technology-based  instruction.  It  also  covers  many  subject 
matters,  posing  questions  following  text  paragraphs,  graphic  displays,  simulations,  audio 
presentations,  video  sequences,  and/or  other  sources  of  instructional  content,  but  the  under¬ 
lying  logic  remains  the  same  as  Crowder’s  original:  display  something,  elicit  a  response, 
and  branch  to  remedial  or  reinforcing  material  depending  on  the  response. 

To  prepare  an  instructional  item,  a  developer  must  anticipate  and  prepare  responses 
for  several  discrete  cognitive  states,  represented  by  the  correct  answer  (response  C)  and  the 
“distractors”  (responses  A,  B,  and  D)  to  the  item.  The  cognitive  model  represented  by 
these  states  is  static,  implicit,  and  limited,  but  it  is  there.  The  main  difference  between  the 
cognitive  modeling  in  Crowder’s  and  Skinner’s  approaches  and  the  cognitive  modeling 
being  developed  today  is  that  the  earlier  models  for  intrinsic  and  extrinsic  programming 
were  implicit,  embodied  in  the  program  of  instruction,  whereas  today  developers  are 
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attempting  to  use  more  explicit  models  of  cognition  that  can  be  abstracted,  expressed,  and 
validated  separately  from  the  systems  in  which  these  models  are  used. 

2.  Explicit  Cognitive  Models:  Quantitative 

These  more  explicit  cognitive  models  are  being  used  for  applications  such  as  intelli¬ 
gent  tutoring  systems  (ITSs)  and  the  human  behavior  modeling  needed  to  generate  com¬ 
puter  (automated)  military  forces  for  constructive  and  virtual  simulation.  Explicit  models  of 
cognition  were  also  applied  early  on  (in  the  1960s).  These  simple  models  were  intended  to 
account  for  rudimentary  learning  objectives  that  could  be  reduced  to  something  like  the  sub¬ 
stantial  amounts  of  stimulus-response,  associative  pairing  required  to  learn  material  such  as 
arithmetic  “facts”  (addition,  subtraction,  multiplication  tables),  second  language  vocabulary, 
and  technical  jargon  (names  and  functions  of  biological  or  mechanical  structures).  None¬ 
theless,  they  led  to  sophisticated  and  effective  instructional  approaches,  and  the  line  of 
research  needed  to  determine  the  full  range  of  learning  situations  and  objectives  to  which 
they  could  be  applied  was  begun  but  left  unfinished  and  is  rarely  found  today. 

As  an  example  of  this  approach  (and  its  use  of  cognitive  models),  consider  the  fol¬ 
lowing  model  of  learning  (adapted  from  Paulson,  1973),  which  attempted  to  account  for  the 
probability  that  a  particular  item  for  a  particular-  learner  would  transition  from  the  unlearned 
state  (U),  to  either  a  short-term  learned  state  (S)  [i.e.,  present  in  working  memory  (WM)]  or 
to  a  long-term  learned  state  (L)  [i.e.,  stored  in  long-term  memory  (LTM)]: 


State  on  Trial  n+1 

L 

S 

U 

P  (correct) 

State 

on 

Trial  n 

L 

1 

0 

0 

1 

S 

c 

1  — c 

0 

1 

U 

a 

b 

1-a-b 

9 

In  words: 

•  If  a  learned  item  (state  L)  is  presented,  then: 
-  With  probability  =  1,  it  stays  there. 
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•  If  an  unlearned  item  (state  U)  is  presented: 

-  With  probability  =  a,  it  will  transition  to  LTM  and  the  learned  state. 

-  With  probability  =  b,  it  will  transition  to  a  short-term  state  (S)  in  WM, 
from  which  it  can  either  be  learned  or  forgotten. 

-  With  probability  =  1-a-b,  it  will  remain  unlearned. 

•  If  an  item  is  in  short-term  WM  (state  S): 

-  With  probability  =  c,  it  will  transition  to  LTM  and  the  learned  state. 

-  With  probability  =  1-c,  it  will  remain  in  the  short-term  state. 

•  An  item  in  the  short-term  state  will  not  slip  back  to  the  unlearned  state. 

This  formulation  accounts  for  guessing.  As  shown  in  the  right-most  column  in  the 
previous  model  of  learning  matrix,  Paulson  assumed  a  probability  =  g  (presumably  for 
“guessing”)  of  a  correct  answer  to  an  unlearned  item  but  assumed  a  probability  =  1  for  a 
correct  answer  to  an  item  in  the  learned  or  short-term  state.  The  parameters  are  estimated  for 
each  item- student  combination. 

A  key  feature  of  this  model  is  that  it  accounts  for  items  that  are  not  presented  on  a 
trial.  In  Paulson's  formulation,  which  is  based  on  Rumelhart’s  General  Forgetting  Theory 
(1967),  when  an  item  is  not  presented,  transitions  between  states  are  expected  to  occur  in 
accord  with  the  following  transition  matrix: 


State  on  Trial  n+1 

L 

s 

u 

State 

on 

Trial  n 

L 

1 

0 

0 

S 

0 

1-f 

f 

U 

0 

0 

1 

In  words,  when  an  item  is  not  presented: 

•  If  it  is  in  the  learned  or  unlearned  state,  it  stays  there. 

•  If  it  is  in  the  short-term  state,  it  may  regress  to  the  unlearned  state  with  prob¬ 
ability  f  or  remain  in  the  short-term  state  with  probability  1-f. 

Formulations  such  as  this,  which  are  based  on  explicit  transition  models  of  memory, 
led  to  an  instructional  strategy  that  has  proven  to  be  optimal  in  maximizing  the  number  of 
items  learned  in  the  total  time  set  aside  for  instruction,  T,  and  allowing  for  a  predetermined 
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number  of  items,  N,  to  be  presented  in  a  single  session  (e.g.,  Atkinson  and  Paulson,  1972). 
The  optimal  solution  determines  which  N  items  to  present  to  a  particular  student  so  that  the 
total  number  of  items  the  student  learns  is  maximized  at  time  T.  The  solution  is  roughly  the 
following: 

•  Before  each  trial,  identify  the  item  or  items  in  N  that  have  received  the  fewest 
number  of  correct  responses  since  the  last  error. 

•  If  only  one  item  is  identified,  present  that  item.  If  more  than  one  item  is  identi¬ 
fied,  select  from  this  group  the  item  or  items  that  have  been  presented  the  few¬ 
est  number  of  times. 

•  If  only  one  item  remains,  present  that  item.  If  more  than  one  item  remains, 
select  one  at  random  and  present  it. 

This  description  does  not  describe  how  items  that  have  reached  criterion  in  the  cur¬ 
rent  pool  of  N  items  can  be  optimally  replaced  with  new  items.  Atkinson  and  Paulson 
(1972)  and  Chant  and  Atkinson  (1973)  have  discussed  such  procedures. 

Quantitative  models  of  this  sort  continue  to  be  used  in  technology-based  instruction, 
as  evidenced  by  efforts  to  apply  Bayesian  networking  to  assess  the  cognitive  states  of  learn¬ 
ers  (e.g.,  Van  Lehn  and  Niu,  2001).  These  models  use  Bayes’  theorem  to  work  backward 
from  users’  responses  to  determine  the  probabilities  that  they  are  using  (perhaps  have 
learned)  specific  cognitive  processes.  This  approach  can  lead  to  quite  sophisticated  models 
of  learners’  knowledge  and  skills. 

Three  points  may  be  worth  making  here: 

1.  Both  implicit  and  explicit  models  of  cognition  and  cognitive  processes  have 
been  used  in  technology-based  instruction  from  its  beginning. 

2.  Fairly  simple  cognitive  models  for  fairly  simple  instructional  paradigms  can 
lead  to  sophisticated  and  effective  instructional  strategies. 

3.  This  approach  remains  a  promising  line  of  quantitative  research  that  deserves  to 
be  explored  more  fully. 

3.  Explicit  Cognitive  Models:  Qualitative 

A  line  of  R&D  in  cognitive  modeling  that  has  been  more  vigorously  pursued  in 
recent  years  is  less  quantitative  than  the  preceding  models,  but  the  range  of  cognition  cov¬ 
ered  tends  to  be  more  comprehensive  and  can  be  used  to  meet  a  wider  range  of  learning 
objectives.  This  work  typically  comes  under  the  heading  of  “human  behavior  modeling” 
and  is  increasingly  used  in  the  development  of  simulations  for  training  personnel  and  units, 
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analyzing  tactical,  operational,  and  strategic  alternatives,  and  designing,  developing,  and 
acquiring  military  materiel. 

We  are  fortunate  that  several  systematic  and  comprehensive  analyses  of  these  mod¬ 
els  have  appeared  recently,  such  as  those  by  Pew  and  Mavor  (1998),  who  reviewed  1 1  such 
models;  Ritter  et  al.  (2002),  who  reviewed  7  models  not  covered  by  Pew  and  Mavor;  and 
Morrison  (2003),  who  reviewed  19  such  models. 

In  these  reviews,  the  models  selected  for  analysis  were  intentionally  devised  to  be 
implemented  in  digital  form  (in  computer  algorithms).  Doing  this  for  any  model  is  a  signifi¬ 
cant  demonstration.  If  a  model  can  be  represented  in  an  algorithm,  it  can  be  tested.  Using  its 
algorithmic  representation  to  capture  and  test  cognitive  processes  can  significantly  enhance 
our  knowledge  of  these  processes  and  the  effectiveness  of  our  education,  training,  and 
performance-aiding  applications.  Diagnostic  information  that  indicates  where  the  model  is 
correct  will  demonstrate  the  validity  of  the  model,  and  diagnostic  information  that  indicates 
where  the  model  is  incorrect  will  suggest  where  it  must  be  modified  to  account  for  the  full 
range  of  human  cognition.  Significant  scientific  and  technological  advances,  as  well  as  sub¬ 
stantial  improvements  in  our  ability  to  educate,  train,  and  assist  military  personnel,  can  arise 
from  this  sort  of  information. 

As  Morrison  (2003)  points  out,  most  of  these  models  are  systems  of  if-then,  condi¬ 
tion-response  (“production”)  rules  that  simulate  cognitive  structures  and  processes. 
Table  1  summarizes  the  19  models  he  reviewed.  These  models  provide  a  snapshot  of  the 
current  state  of  human  cognition  and  behavior  representation. 

How  can  these  models  contribute  to  the  development  of  computer-mediated  learning 
and  performance-aiding  environments?  As  suggested  previously,  a  model  intended  to  sup¬ 
port  education  and  training  needs  either  an  implicit  or  explicit  model  of  cognition  if  it  is  to 
assess  the  state  of  a  learner’s  knowledge,  skill,  and  abilities.  To  do  this,  it  must  represent 
memory  and  its  interactions  with  other  cognitive  functions,  such  as  perception  and  attention. 
It  can  also  represent  cognitive  functions  such  as  decision-making  and  problem  solving  and 
cognitive  responses  to  the  environment,  such  as  social  behavior  and  situation  awareness 
(SA)  and/or  the  extent  of  cognitive  workload. 

However,  if  a  model  is  developed  to  support  education  and  training,  having  it  simply 
represent  the  current  state  of  cognitive  processing  is  not  enough.  It  must  also  represent  and 
project  its  evolution  and  development.  In  short,  it  must  include  a  model  of  human  learning. 
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Table  1.  Summary  Descriptions  of  Cognitive  Aspects 
in  Models  Reviewed  by  Morrison  (Source:  Morrison,  2003) 


Model  Name 

Summary  Description 

Reference(s) 

Atomic  Components  of 
Thought  (ACT) 

Intended  to  provide  a  unified  theory  of  mind  and  a 
design  basis  for  instructional  environments  (e.g., 
intelligent  tutors,  computer-generated  forces)  and 
human  interfaces.  Distinguishes  between  declarative 
knowledge  (represented  with  semantic  networks)  and 
procedural  knowledge  (represented  using  if-then  rules). 

Lebiere,  2002 

Anderson  et  al. , 

2002 

Adaptive  Resonance  Theory 
(ART) 

Family  of  neural  net  models  designed  to  explain 
sensory-cognitive  processes  (e.g.,  perception, 
recognition,  attention,  reinforcement,  recall,  and  WM). 
Postulates  bottom-up  (e.g.,  perceptions)  and  top-down 
(e.g.,  expectations,  attention  control)  functions  in  WM 
that  interact  to  produce  learning. 

Grossberg,  1976a; 
1976b 

Krafft,  2002 

http://web.umr. 

edu/~tauritzd/art 

Architecture  for  Procedure 
Execution  (APEX) 

Intended  to  reduce  the  time  and  effort  needed  to 
develop  models  of  human  performance  in  complex, 
dynamic  environments  (e.g.,  simulations,  explorations 
of  human  performance  theories,  and  assessments  of 
equipment  design  on  human  performance).  Includes 
goal-directed  action  selection  for  tasks  and  procedures 
and  resource  allocation  for  perceptual  (mostly  visual), 
cognitive,  and  psychomotor  functions. 

Freed  et  al.,  2002 

http://www. 

andrew.cmu.edu/~ 

bj07/apex 

Business  Redesign  Agent- 
Based  Holistic  Modeling 
System  (Brahms) 

Models  social  and  man-machine  interactions.  Uses 
agents  to  model  interactions  among  physically 
dispersed  groups  (e.g.,  teams),  and  if-then  rules 
(“detectables”  and  “beliefs”)  to  model  decision-making 
(via  “thoughtframes”)  and  behavior  within  the  groups. 
Emphasizes  ethnographic  analyses  and  socio- 
technical  work  practices,  activities  shaped  by  socio- 
technical  environment,  and  constructivist,  situated 
cognition  to  model  cognition  and  behavior. 

Sierhuis  and 

Clancey,  1997 

Clancey  et  al., 

1998 

Acquisti  et  al., 

2001 

Cognition  and  Affect  Project 
(CogAff)  (with  associated 
SimAgent  toolkit) 

Conceptual  space  for  describing  cognitive 
architectures.  Integrates  emotional  and  cognitive 
processes.  Incorporates  three  layers  of  cognition 
(reactive,  deliberative,  and  reflective  or  meta- 
cognitive),  three  layers  of  information  processing 
(perception,  central  processing,  and  action),  and  three 
types  of  emotions  (primary  based  on  reaction, 
secondary  based  on  deliberation,  and  tertiary  based  on 
reflection)— all  producing  different  perceptual,  memory, 
and  motor  functions. 

Sloman,  2001 ; 

2003 

http://www.es. 

bham.ac.uk/~axs/c 

ogaff.html 
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Table  1.  Summary  Descriptions  of  Cognitive  Aspects 
in  Models  Reviewed  by  Morrison  (Source:  Morrison,  2003)  (Continued) 


Model  Name 

Summary  Description 

Reference(s) 

Cognition  as  a  Network  of 
Tasks  (COG NET)  [with 
associated  Generator  of 
Interface  Agents  (GINA)  and 
iGEN™  toolkits) 

Intended  for  cognitive  task  analysis  and  description  of 
work  domains  in  multitask  environments  requiring 
contemplative,  decision-oriented,  open-ended 
responses.  Uses  three  subsystems  to  represent 
information  processing  (sensory/perceptual,  mental 
modeling,  action/motor),  four  forms  of  if-then  rule- 
based  task  knowledge  (goal-directed  task  hierarchies, 
perceptual  demons  to  guide  attention,  blackboard  for 
organizing  declarative  information,  and  possible 
actions  linked  to  time  and  resource  requirements),  and 
meta-cognitive  functions.  Allows  interfacing  with  other 
applications. 

Zachary  et  al., 

2001 

http://www.chiinc. 

com/cognethome.s 

html 

Cognitive  Complexity  Theory 
(CCT)  (with  associated 

GOMS  Language  and 
Evaluation  Analysis-3 
(GLEAN3)  toolkit) 

Focuses  on  human  interface  design,  human-computer 
interaction,  and  sequential  task  performance.  Employs 
device  models  (transition  networks),  user  models 
(sequentially  executed  if-then  rules,  the  fundamental 
CCT  units  of  cognition,  retrieve  from  LTM),  and  mental 
operators  to  represent  covert  cognitive  processes. 

Kieras  and  Poison, 

1985 

Kieras,  1999 

Cognitive  Objects  within  a 
Graphical  EnviroNmenT 
(COGENT) 

Intended  solely  to  provide  tools  (via  a  visual 
programming  environment  that  evolves  with  the  model 
being  built)  for  cognitive  modeling,  assuming  functional 
modularity  (cognition  as  interaction  among  semi- 
autonomous  subsystems)  and  using  low-level 
processing  components. 

Cooper,  Yule,  and 
Sutton,  1998 

Yule  and  Cooper, 
2000 

http://cogent. 

psyc.bbk.ac.uk 

Concurrent  Activation- 
Based  Production  System 
(CAPS) 

Hybrid  model  for  central  cognitive  functions  (e.g., 
reading  comprehension).  Primary  focus  is  on  modeling 
brain  activation  patterns  in  high-level  cognition  via  if- 
then  rules  for  specific  areas  of  the  brain  and 
associative  networks  for  cognitive  subsystems.  Total 
activation  in  WM  is  capped,  concerned  exclusively  with 
declarative  knowledge  (facts),  but  with  different  limits 
for  different  individuals.  LTM  includes  procedural  and 
declarative  knowledge. 

Just,  Carpenter, 
and  Varma,  1999 

http://coglab.psy. 

cmu.edu/projects_ 

set.html 
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Table  1.  Summary  Descriptions  of  Cognitive  Aspects 
in  Models  Reviewed  by  Morrison  (Source:  Morrison,  2003)  (Continued) 


Model  Name 

Summary  Description 

Reference(s) 

Construction-Integration 
Theory  (C-l  Theory) 

Uses  a  symbolic  theory  of  sentence  comprehension 
and  propositions  (actions  and  objects  of  the  action)  and 
stresses  goal  formation  to  provide  a  general  model  of 
cognition.  Comprehension  progresses  from 
approximations  to  verified  integration  through  mutually 
reinforced  associations  and  spreading  activation  in 
memory.  Extended  to  cover  comprehension  of  novel 
computer  interfaces  [Linked  model  of  Comprehension- 
based  Action  planning  and  Instruction  (LICAI)]  and  new 
Web  sites  [Comprehension-based  Linked  model  of 
Deliberate  Search  (CoLiDeS)  model]  and  to  incorporate 
concepts  from  Latent  Semantic  Analysis  (LSA)  used  to 
derive  meaning  from  text. 

Kintsch,  1998 

Landauer  and 

Dumais,  1997 

Kitajima  and 

Poison,  1997 

Kitajima, 

Blackmon,  and 
Poison,  2000 

http  ://psych- 
www. Colorado. 

edu/ics 

Distributed  Cognition 
(DCOG) 

Intended  to  model  individuals’  expert  behavior  with 
agents  that  use  multiple  strategies  to  respond  to  a 
complex  environment  (air  traffic  control).  Based  on  a 
two-dimensional  (2-D)  space:  abstraction  with  three 
levels  (skill-based  responses  to  signals,  rule-based 
responses  to  signs,  and  knowledge-based  responses 
to  symbols)  and  decomposition  (ranging  from  individual 
component  to  total  system  processing).  Processing 
within  this  space  depends  on  the  level  of  expertise,  the 
workload  environment,  and  an  individual’s  preferred 
level  of  engagement. 

Eggleston,  Young, 
and  McCreight, 

2000; 2001 

Executive  Process/ 

Interactive  Control  (EPIC) 

Intended  to  model  details  of  peripheral  cognitive 
processes,  input  (perception),  and  output 
(psychomotor  responses)  to  inform  human-system 
interface  design  by  predicting  the  order  and  timing  of 
responses.  Includes  long-term  storage  of  declarative 
and  procedural  knowledge  and  WM  for  assessing  their 
application.  Capacity  and  retrieval  limitations  arise  only 
from  perceptual  and/or  psychomotor  systems,  not  from 
central  memory  store. 

Kieras  and  Meyer, 
1995 

http://www.eecs. 

umich.edu/~kieras/ 

epic.html 

Human  Operator  Simulator 
(HOS) 

Intended  to  inform  human-system  interface  design  by 
modeling  human  performance  based  on  the  sequence 
and  timing  of  subtasks  organized  in  networks.  Uses 
simulation  objects  (configuration  of  displays  and 
controls),  task  networks  (if-then  rules  selecting  verb- 
object  pairs  used  to  manipulate  the  objects),  and  micro¬ 
models  (times  to  complete  required  subtasks  involving 
perception,  information  processing,  and  psychomotor 
responses)  to  determine  human  response  times. 

Wherry,  1976 

Harris,  lavecchia, 
and  Dick,  1989 

Glenn,  Schwartz, 
and  Ross,  1 992 
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Table  1.  Summary  Descriptions  of  Cognitive  Aspects 
in  Models  Reviewed  by  Morrison  (Source:  Morrison,  2003)  (Continued) 


Model  Name 

Summary  Description 

Reference(s) 

Man-machine  Integrated 
Design  and  Analysis  System 
(MIDAS) 

Intended  to  inform  human-system  interface  design  by 
modeling  individuals  and  interactions  among  individuals 
in  performing  multiple,  concurrent  tasks.  Uses  sensory 
input  [operators  and  perceivable  (detectable, 
recognizable,  and  identifiable)  objects],  memory  [with 
declarative  (beliefs  in  LTM,  contexts  in  WM)  and 
procedural  components],  decision-making,  attention 
(with  limitations  on  processing  resources),  SA  (actual 
and  perceived),  and  psychomotor  output  to  model 
human  operator  limitations  and  capabilities. 

Corker  and  Smith, 

1993 

Hart  et  al.,  2001 

http://caffeine.arc. 

nasa.gov/midas/in 

dex.html 

Micro  Systems  Analysis  of 
Integrated  Network  of  Tasks 
(Micro  SAINT)  [may  include 
the  Integrated  Performance 
Modeling  Environment 
(IPME),  using  HOS  micro¬ 
models,  and  WinCrew  for 
estimating  workload] 

Simulation  tool  that  uses  a  detailed  task  analysis  to 
decompose  human  performance  into  a  networked 
hierarchy  (with  branching  logic  and  sequential 
dependencies)  of  discrete  tasks  and  subtasks  for 
which  performance  estimates  can  be  validated. 

Network  consists  of  subtask  nodes  (with  launching 
conditions,  time  to  complete,  and  effects)  and 
relationships  (that  may  be  probabilistic,  tactical 
requiring  a  threshold  value,  or  multiple  initiating  more 
than  one  subtask).  Designed  to  communicate  with  other 
models  and  applications  through  middleware. 

Laughery  and 

Corker,  1997 

Operator  Model  Architecture 
(OMAR)  (Flavors  Expert 
(FLEX),  an  expert  system 
toolkit,  to  model  decision¬ 
making  as  a  rule-following 
process) 

Models  human  behavior  as  interactions  among 
independent  computational  agents  representing 
interacting  individuals  or  cognitive  processes  within 
individuals.  Allows  both  sequentially  dependent  and 
parallel  task  performance,  with  order  determined  by 
activation  levels  of  tasks  (without  an  explicit  executive 
process).  Allows  facile  interface  with  other  models. 

Deutsch, 

MacMillan,  and 
Cramer,  1993 

Deutsch,  1998 

Cramer,  1998 

PSI  (Not  an  acronym)1 

Attempts  to  integrate  motivation  with  cognitive 
processes.  Based  on  three  levels  of  needs  that  interact 
to  determine  motive  strength  and  specific  goal 
behaviors:  system  needs  (water  and  energy), 
preservation  level  (pain  avoidance),  and  information 
level  (certainty,  competence,  affiliation).  Action 
strategies  first  seek  automatized  skills,  then 
knowledge-based  behavior,  and  then  trial  and  error  to 
satisfy  goals. 

Bartl  and  Dorner, 

1998 

Ritter  et  al.,  2002 

http://www.uni- 

bamberg.de/~ba2d 

pl/psi.html 

*  PSI  is  usually  presented  in  all  capital  letters,  but  has  not  been  defined  as  an  acronym.  According  to 
Dietrich  Domer,  “PSI  is  not  an  acronym.  It  just  is  the  first  letter  of  the  Greek  word  for  "soul".  And 
this  is  because  it  is  our  intention  with  the  PSI-project  not  only  to  simulate  cognition,  but  motivation, 
emotion  and  what  we  call  “action  regulation”  too.  Just  the  whole  soul!  And  that's  the  idea  behind  PSI” 
(personal  communication,  December  29,  2003). 
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Table  1.  Summary  Descriptions  of  Cognitive  Aspects 
in  Models  Reviewed  by  Morrison  (Source:  Morrison,  2003)  (Continued) 


Model  Name 

Summary  Description 

Reference(s) 

Situation  Awareness  Model 
for  Pilot-in-the-Loop 

Evaluation  (SAMPLE) 

Generalized  from  original  effort  to  model  SA  of  pilots 
and  air  crews  in  air  combat.  Uses  cognitive  task 
analyses,  pattern  recognition  from  Klein’s  Recognition- 
Primed  Decision-Making,  Endsley’s  three  levels  of 
awareness  (detection,  identification,  and  prediction), 
and  Rasmussen’s  three  tiers  of  action  strategy  (skill- 
based  pattern  recognition,  standardized  if-then  rules, 
and  knowledge-based  problem  solving)  to  provide  three 
stages  of  processing:  information  processing  (with  a 
continuous  state  estimator  and  a  discrete  event 
detector),  situation  assessment  (with  the  information 
fusion  and  reasoning  required  by  multitasking),  and 
decision-making  (with  a  procedure  selector  and  a 
procedure  executor).  Output  includes  information 
disparity,  SA  disparity,  and  combat  advantage  index. 

Rasmussen,  1983 

Endsley,  1988 

Klein,  1989 

Mulgund,  Harper, 
and  Zacharias, 

2000 

State,  Operator,  And  Result 
(Soar) 

Intended  as  a  comprehensive  model  of  human  cognition 
focused  on  operational  task  domains  depicting  all 
behavior  as  goal-driven  movement  through  problem 
spaces  that  define  states  and  operators  for  the  task(s) 
at  hand.  Uses  a  four-cycle  iterative  process  involving 
input  (via  human  perception),  elaboration  (matches  if- 
then,  condition-action  rules  in  LTM  with  those  in  WM  to 
issue  proposals  for  decision-making  and  direct 
commands  for  psychomotor  actions),  output 
(psychomotor  execution),  decision  (either  selects 
operators  or  identifies  “impasses”  requiring  a  new 
subgoal  until  all  impasses  are  resolved).  Uses  a  single 
process  for  LTM,  learning,  task  representation,  and 
decision-making.  All  learning  occurs  through 
“chunking,”  which  occurs  through  impasse  subgoaling 
and  resolution.  Emotions  arise  from  SA  clarity  and 
confusion.  Integrates  individual  and  team  knowledge 
and  allows  goals  and  plans  to  be  shared  among  team 
members. 

Lewis,  2001 

http://ai.eecs. 

umich.edu/soar 

http://www- 
2. cs.cmu.edu/afs/ 
cs/project/ 
soar/public/www/ 
home-page.html 

http://www.isi.edu/ 

soar/soar- 

homepage. 

html 

http://www. 

nottingham.ac.uk/p 

ub/soar/ 

nottingham/ 

soar-faq.html 

http://phoenix. 
herts.ac.uk/~rmy/ 
cogarch. seminar/ 
soar.html 

Table  2,  taken  directly  from  Morrison  (2003),  summarizes  the  cognitive  functions  covered 
by  the  models  summarized  in  Table  1.  It  indicates  which  models  explicitly  represent  one  or 
more  of  the  following  cognitive  processes:  perception,  psychomotor  performance,  attention, 
SA,  short-term  memory  (STM),  LTM,  learning,  decision-making,  problem  solving,  cognitive 
workload,  emotional  behavior,  and  social  behavior. 


14 


Table  2.  Cognitive  and  Behavioral  Functions  Represented 
in  Models  Reviewed  by  Morrison  (Source:  Morrison,  2003) 


Acronym/ 

Abbreviation 

ACT 

ART 

APEX 

Brahms 

CogAff 

COGNET 

CCT 

COGENT 

CAPS 

C-l  Theory 

DCOG 

EPIC 

HOS 

MIDAS 

Micro  SAINT 

OMAR 

PSI 

SAMPLE 

Soar 


Cognitive  Function  Required 


Perception 

Psychomotor 

Performance 

Attention 

Situation 
Awareness  (SA) 

Short-term 

Memory  (STM) 

Long-term 

Memory  (LTM) 

Learning 

Decision-Making 

Problem  Solving 

Cognitive 

Workload 

Emotional 

Behavior 

Social  Behavior 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X 

X  X  X  X 


X  X  X  X  X 


X 


X  X  X  X  X  X  X  X 


X 


Table  2  indicates  that 

•  All  19  models  represent  decision-making,  but  it  is  largely  the  reactive  form  of 
decision-making  captured  in  if-then  rules. 

•  All  19  models  represent  either  STM  or  LTM. 

•  Perception  and  attention  were  well  represented  in  16  of  the  reviewed  models. 
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Social  Behavior 


•  Although  only  four  of  the  models  explicitly  represented  SA,  the  functions  of 
SA  were  present  in  those  representing  perception  and  attention. 

•  Social  behavior  was  represented  in  only  five  of  the  models. 

•  Emotional  behavior  was  represented  in  only  three  of  the  models. 

•  Learning  and  problem  solving  were  represented  in  only  five  of  the  models. 
Morrison  suggests  that  this  limited  representation  may  be  caused  by  the  nature 
of  condition-response  production  models,  which  can  react  to  the  situations 
contained  in  anticipated  if-states  but  which  may  not  adapt  well,  if  at  all,  to  the 
unanticipated  states  and  conditions  that  must  be  accommodated  in  learning  and 
problem  solving. 

The  five  models  judged  to  represent  learning  are  ACT,  COGENT,  CAPS,  PSI,  and 
Soar.  All  five  of  these  models  also  represent  LTM,  STM,  and  decision-making.  All  five  of 
these  models,  except  COGENT,  also  represent  perception,  psychomotor  performance,  and 
attention. 

A  model  of  cognition  that  includes  learning  is  necessary  for  education  and  training 
applications,  but  it  is  not  sufficient.  A  model  of  learning  is  not  a  model  of  instruction.  All 
19  models,  as  good  as  many  of  them  are,  lack  a  model  of  instruction.  This  component  is 
needed  to  suggest  links  between  specific  instructional  interventions  and  specific  learning 
outcomes— teaching  strategies  that  reliably  bring  about  transitions  from  the  learner’s  cur¬ 
rent  cognitive  state  to  one  capable  of  producing  the  intended  instructional  outcomes. 

D.  INSTRUCTIONAL  SYSTEMS  DEVELOPMENT 

Attaining  a  “model  of  instruction”  that  is  centered  around  models  of  human  cogni¬ 
tion  would  lead  to  what  might  be  called  “engineering  of  instruction”— instruction  viewed 
as  neither  art  nor  science,  but  as  a  way  to  produce  specified  instructional  outcomes  reliably 
and  efficiently.  Such  a  capability  for  development  of  instructional  and  performance-aiding 
systems  should  be  based  on  empirically  derived  principles  that  can  be  applied  realistically. 
Outcomes  might  consist  of  general  objectives,  such  as  the  ability  to  transfer  knowledge, 
long-term  retention  of  knowledge  and  skill,  motivation  to  continue  learning,  speed  of 
response,  accuracy  of  response,  and  so  forth.  The  outcomes  might  be  associated  with  more 
specific  training  objectives,  such  as  the  ability  to  locate  single  component  failures  in  the 
XYZ  power  supply,  pack  a  reserve  parachute,  or  devise  tactical  plans. 

Fragments  of  such  a  capability  for  engineering  instruction  have  been  identified  in 
research  literature,  data,  and  findings.  Work  is  needed  to  organize,  substantially  expand,  and 
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include  them  as  principles  to  be  incorporated  in  current  models  of  cognition.  In  addition,  the 
engineering  of  instruction  requires  — as  an  essential  foundational  element— robust  human 
cognitive  models  to  enable  the  training,  education  or  performance-aiding  system  to  “know” 
the  user  and  to  adapt  dynamically  to  the  user’s  state. 

1.  What  R&D  Do  We  Need? 

This  brief  review  of  cognitive  models  applied  to  automated  instructional  and 
performance- aiding  systems  suggests  that  good  progress  has  been  made  but  that  much 
remains  to  be  done.  The  models  needed  to  support  fully  the  broad  range  of  human  behavior 
required  for  simulations  now  used  in  training,  analysis,  and  acquisition  are  not  available. 
More  generally,  still  lacking  are  the  comprehensive  models  needed  to  represent  subject 
matter  expertise,  levels  of  student  learning,  and,  most  especially,  the  links  between  specific 
instructional  interventions  and  the  development  of  specifically  targeted  cognitive  abilities 
needed  for  competent  performance. 

The  question,  therefore,  is:  What  R&D  should  be  pursued  to  achieve  short-,  mid-, 
and  long-term  enhancements  in  the  state  of  the  art?  This  issue  was  addressed  in  a  November 
1999  workshop  that  assessed  the  R&D  needed  to  support  the  Department  of  Defense 
(DoD)  Advanced  Distributed  Learning  (ADL)  initiative  (Final  Report,  1999),  in  a  series  of 
workshops  sponsored  in  2002-2003  by  the  Learning  Federation  (Learning  Federation, 
2003),  and  in  another  human  factors  modeling  (HFM)  symposium  (Foster  and  Fletcher, 
2002).  These  three  sources  cover  a  wide  range  of  issues  and  organize  their  results  in  differ¬ 
ent  categories,  but  some  common  findings,  specifically  those  concerned  with  research  nec¬ 
essary  for  the  development  of  cognitive  models,  emerge  from  them.  Table  3  summarizes 
findings  concerning  cognitive  modeling.  It  presents  these  findings  as  issues  and  indicates 
some  specific  research  needed  to  meet  these  goals  and  fill  gaps  in  our  current  capabilities. 

The  efforts  suggested  by  Table  3  are  realistic  because  they  are  amenable  to  research 
that  can  be  performed  with  approaches  available  from  our  current  state  of  knowledge.  They 
suggest  goals  that  can  be  achieved  to  an  appreciable  degree  in  the  next  3  to  5  years.  Doing 
so  will  be  worth  the  effort  and,  for  the  success  of  our  operational  capabilities,  the  return  on 
investment  will  outweigh  the  cost. 

The  value  of  cognitive  models  has  another  increasingly  important  dimension.  The 
current  world  environment  presents  significant  challenges  to  our  capabilities  for  preparing 
military  personnel  to  meet  these  challenges  and  thereby  to  our  capabilities  for  providing 
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Table  3.  Issues  and  Research  Requirements  for  the  Development  of  Cognitive 
Modeling  Summarized  From  Assessments  of  Learning  Technology  Needs 


Issue 

Research  Requirements 

Cognitive  theory 

•  Representation  of  “higher  order”  cognitive  capabilities  (e.g.,  decision-making, 
problem  solving,  meta-cognition,  pattern  recognition,  critical  thinking,  situational 
awareness,  teamwork) 

•  New  concepts  and  theories  of  cognition  and  cognitive  workload  based  on  new 
measurement  capabilities 

•  Valid  and  verified  representation  of  expertise  and  its  development  in  complex,  ill- 
structured  environments 

•  Knowledge  representations  and  ontologies  that  allow  interoperability  and  logical 
operations  within  and  across  disciplines 

Human  behavior 

representation 

(HBR) 

•  Comprehensive  and  accurate  representation  of  individual  and  of  crew,  team,  and  unit 
expertise,  capabilities,  and  performance 

•  Free,  cognitively  transparent  exchange  of  virtual  (avatar)  and  actual  users  in  crew, 
group,  and  team  learning 

Cognitive  model 
authoring 

•  Automated  development,  verification,  and  validation  of  cognitive  models 

•  Automated  processes  for  performing  cognitive  analysis  and  cognitive  readiness 
assessment 

•  Automated  capture  of  expertise:  self-generating,  self-modifying  databases  built  from 
cases  and  examples  of  successful  problem  solving  and  decision-making 

•  Principles  for  developing  physically  and  cognitively  realistic  avatars 

User  assessment 
and  representation 

•  New  forms  of  computer-administered  assessment  items  using  the  full  display,  timing, 
and  natural  language  understanding  capabilities  of  technology 

•  Generation  of  valid,  unobtrusive  near-real-time  assessment  from  interactions  of 
individuals,  teams,  crews,  and  units  with  the  learning  or  performance-aiding 
environment 

•  Representation  of  subject  matter  misunderstandings  and  their  sources 

•  Generation  and  use  of  questions  to  build  cognitive  profiles  of  users 

•  Assessment  of  cognitive  workload 

•  Assessment  of  the  high-level  cognitive  skills  needed  to  deal  with  unanticipated  and 
unexpected  situations 

Management  of 
progress 

•  Ability  to  match  instructional  or  problem-solving  goals  with  current  state  of  the  user 
and  to  generate  or  select  optimal  tutorial  and/or  problem-solving  strategies 

•  Automated  principles  of  design  and  presentation  needed  to  ensure  reliable 
achievement  of  a  targeted  cognitive  state(s)  by  individuals,  crews,  teams,  and  units 

•  Automated  principles  for  the  development  of  higher-level  cognitive  skills  such  as 
creativity,  adaptability,  problem  solving,  and  SA 

•  Comprehensive  understanding  of  meta-cognition  and  its  development 

•  Comprehensive  understanding  of  incentive  management  and  its  interaction  with 
cognitive  development 

•  Technology-based  tools  allowing  distributed  users  to  manage  their  own  progress  and 
problem  solving 

•  Predictions  of  learning  rate  and  success  from  user  profile  information 

User  interface 

•  Management  of  user  dialogue  based  on  model  of  user  cognitive  abilities,  style,  and 
progress  toward  objective(s) 
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military  education  and  training.  We  have  responded  in  ways  that  have  proven  successful  in 
the  past,  with  task  lists,  essential  task  lists,  mission-essential  task  lists,  and  even  joint  mis¬ 
sion-essential  task  lists.  These  task  lists  suggest  education  and  training  objectives  that  we 
know  how  to  meet. 

2.  Preparing  for  the  Unexpected 

The  current  asymmetric,  unpredictable  operational  environment  now  facing  our 
military  personnel  will  inevitably  present  situations  that  are  unexpected  and  for  which  they 
may  not  be  fully  prepared.  Our  military  and  our  allies  will  have  to  respond  to  these  situa¬ 
tions  with  agility,  flexibility,  creativity,  and  skillful  leadership.  Their  readiness  to  acquire  the 
additional  capabilities  needed  to  meet  these  unexpected,  unforeseen  challenges  will  contrib¬ 
ute  substantially  to  the  success  of  their  operations.  How,  then,  can  we  best  prepare  our  peo¬ 
ple  to  expect  the  unexpected  and  deal  with  it  successfully?  One  important  aspect  of  this 
readiness  is  a  cognitive  capability.  It  places  special  demands  on  our  ability  to  model  cogni¬ 
tion  and  to  train  individuals  and  units.  It  is  an  essential  component  of  what  we  have  called 
cognitive  readiness  (Etter,  Foster,  and  Steele,  2000),  and  a  combination  of  technology-based 
education,  training,  and  performance  aiding  is  expected  to  help  our  forces  achieve  it. 

The  components  of  cognitive  readiness  cover  issues  that  include  the  following: 

•  SA.  This  is  generally  defined  as  the  ability  to  perceive  oneself  in  relation  to  the 
enemy  and  the  environment.  SA  has  been  shown  to  improve  with  practice  and 
instructional  feedback. 

•  Memory.  This  is  described  as  an  active,  reconstructive  process  supported  by 
two  underlying  theoretical  mechanisms:  encoding  specificity,  which  stresses  the 
importance  of  external  and  internal  cues,  and  transfer-appropriate  processing, 
which  stresses  actions  performed  during  encoding  and  retrieval.  Tradeoffs  exist 
between  instruction  used  to  enhance  the  retention  and  speed  of  initial  acquisi¬ 
tion.  Conditions  of  learning,  particularly  those  providing  overlearning,  can  be 
designed  to  improve  retention. 

•  Transfer  of  training.  This  is  described  as  the  ability  to  apply  what  is  learned 
in  one  performance  context  to  another  performance  context.  Massive  amounts 
of  practice  with  feedback  will  improve  “low-road”  transfer  and  require  little 
cognitive  mediation.  Training  in  forming  mindful,  conscious  abstraction  will 
improve  “high-road”  transfer,  which  requires  cognitive  mediation. 

•  Metacognition.  This  refers  to  the  executive  functions  of  thought,  particularly 
those  pertaining  to  knowledge  and  regulation  of  one’s  cognitive  processes  and 
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progress  toward  accepted  goals.  Metacognitive  skills  can  be  improved  by  exer¬ 
cises  designed  to  increase  the  awareness  of  self-regulatory  processes. 

•  Automaticity.  This  refers  to  processes  that  are  performed  rapidly,  requiring 
few  attentional  resources.  Practice  with  feedback  and  overlearning  can  produce 
automatic  processing  in  many  tasks. 

•  Problem  solving.  This  transforms  goals  and  subgoals  into  a  plan  of  action  by 
processes  such  as  trial  and  error,  proximity,  fractionation,  and  knowledge-based 
referrals.  Techniques  for  problem  solving  matched  to  goal  and  situation  catego¬ 
ries  can  be  successfully  taught,  as  can  the  information  base  needed  for 
“strong”  problem-solving  methods,  which  depend  on  acquired  knowledge. 

•  Decision-making.  This  is  described  as  the  selection  of  tactical  and  strategic 
plans,  which  are  frequently  primed  by  the  recognition  of  learned  patterns.  For¬ 
mal  instruction  in  decision-making  techniques  may  improve  the  quality  of  deci¬ 
sions,  but  some  aspects  of  successful  decision-making  are  determined  by 
individual  dispositions. 

•  Mental  flexibility  and  creativity.  This  can  be  cast  as  problem  solving.  It 
applies  “strong”  methods,  which  are  based  on  acquired  knowledge  and  skills, 
and  “weak”  methods,  which  are  used  for  poorly  defined,  ill-structured,  chaotic 
tasks.  Creativity  may  be  more  closely  associated  with  the  “weak”  methods. 
The  ability  to  train  these  weak  methods  directly  is  unclear  from  the  research.  It 
seems  more  likely  that  native  abilities  determine  the  facility  with  which  people 
apply  appropriate  weak  methods  (i.e.,  achieve  “creative  solutions”)  to  novel 
situations. 

•  Leadership.  This  appears  to  consist  of  motivational  patterns  and  a  combina¬ 
tion  of  technical,  conceptual,  and  interpersonal  skills— the  last  being  the  most 
difficult  to  acquire  and  measure.  However,  the  technical  and  conceptual  skills 
needed  by  leaders  can,  to  an  appreciable  extent,  be  taught.  Interpersonal  skills 
and  patterns  of  motivation  required  for  leadership  appear  to  be  more  dependent 
on  native  abilities  and  are  thus  more  difficult  to  teach. 

•  Emotion.  This  must  be  channeled  and  controlled  if  military  personnel  are  to 
perform  complex  tasks  under  the  stress  and  confusion  that  accompany  modem 
military  operations.  Deeply  engaging,  sensory-immersing  simulations  provide 
promise  for  training  warfighters  to  retain  critical  pieces  of  information  and  to 
perform  under  highly  stressful  conditions. 

These  issues  have  been  discussed  extensively  in  research  literature,  and  Morrison 
and  Fletcher  (2002)  have  discussed  their  specific  relevance  to  cognitive  readiness.  The 
points  to  be  emphasized  are  that 
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•  The  assessment  and  development  of  the  capabilities  suggested  by  these  issues 
will  key  on  the  adequacy  of  the  cognitive  models  on  which  our  education, 
training,  and  performance  aiding  are  based. 

•  The  adequacy  of  our  cognitive  modeling  is  a  matter  of  first  importance  in  the 
current  unpredictable  operational  environment. 

E.  SUMMARY 

The  modeling  efforts  reviewed  in  this  document,  along  with  similar  efforts  involving 
human  cognition,  represent  significant  opportunities  for  cooperative  research  by  the  North 
Atlantic  Treaty  Organization  (NATO)  community  concerned  with  the  human  competence 
that  is  an  essential  component  of  every  military  operation.  We  should  respond  to  the 
opportunities  they  present. 
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